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Predicting Counselors' Test Use Practices 

Because testing is an important part of the work of many 
counselors, nearly all counselor training programs require course 
work in measurement. For example, 98 percent of training programs 
for school counselors require one cr more courses in measurement 
(Schafei and Lissitz, 1987). 

However, it is unclear if this preparation is sufficient to 
ensure good test use practices. Lambert (1991) has asserted that 
there is a crisis in measurement literacy in psychology and 
education. She argues that courses in individual testing, program 
evaluation, or statistics are insufficient to provide a knowledge 
of psychometric theory necessary to select and use tests 
appropriately. 

Professional organizations and other groups have been 
involved in a variety of efforts to help counselors become better 
informed about appropriate test use. For example, the American 
Association for Counseling and Development has a section on 
measurement and evaluation as part of its Ethical standards 
(AACD, 1988) ; this organization has also produced materials to 
guide counselors in responsible test use (AACD/ANECD, 1989) . The 
Joint Committee on Testing Practices, which has included 
representatives from AERA, APA, AACD, NCME and AS HA, has 
published the Code of Fair Testing Practices in Education (Joint 
Committee, 1988). The Test User Qualification Working Group has 
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developed a model of the knowledge, skills, and abilities needed 
to prevent test misuse (Eyde et al., 1988). 

However, little Is known about the extent to which 
counselors Incorporate In their work the basic principles of good 
testing practice, whether learned in pre -service education, in- 
service training, or from reading these ethical standards and 
guidelines. This presentation reports the results from a survey 
of the test use practices of counselors who belong to either or 
both of two divisions of the American Association for Counseling 
and Development, the American School Counselors Association 
(ASCA) and the Association for Measurement and Evaluation in 
Counseling and Development (AMECD) . The goals of the survey 
were: 1) to ascertain current test use practices among 
counselors, and 2) to explore the relationship between education, 
training, measurement knowledge and test use practices. A 
particular interest was determining if familiarity with test use 
guidelines is associated with good test use practices. 
Methods. 

A questionnaire covering test use patterns; test 
administration, selection, and interpretation practices; and 
acquaintance with test use guidelines was developed. The 
questionnaire was sent to a sample of 2,219 individuals, 
including all persons who were, as of June 1989, members both of 
ASCA and AMECD (N»334) and to a random sample of ASCA members 
(N-1192) and AMECD members (N-693). Individuals living outside of 
the United States and individuals who are students were excluded 
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from the sample, one month after the initial mailing, a follow- 
up postcard was sent to non-respondents. Seven hundred thirty 
nine questionnaires were returned (a 33 percent response rate) . 
Six hundred seventy two usable responses are the basis of this 
analysis. 

Because of the comparatively low response rate, an analysis 
comparing respondents and non-respondents was undertaken. 
Respondents were found to differ from non-respondents in 
race/ethnicity (a higher proportion of Whites than minority group 
members responded) , gender (more males than females responded) , 
work setting and job title (fewer individuals employed in schools 
and fewer counselors responded) , and highest degree (more 
individuals with doctorates responded) . Therefore, the data in 
this survey may not be representative of ASCA and AMECD members. 

A descriptive analysis computed the percentage of 
respondents selecting the response and, also, the means and 
standard deviations for each questionnaire item. In addition two 
research questions were posed: 

1) What counselor background, education, and work situation 
variables are associated with having read the Code of Fair 
Testing Practices in Education (Code) and/or the Responsibilities 
of Users of standardized Tests (RUST)? 

2) What counselor background, education, knowledge and work 
situation variables are associated with good test use practices? 

Both questions were answered by use of ordinary least 
squares regression analysis. The regression to answer Question 1 
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is based on the responses of all subjects who have any type of 
test use responsibility. The analyses about good test use 
practices in test selection, test administration, and test 
interpretation are United to subjects who have responsibilities 
in the particular area. 

The conceptual models for these regressions are shown in 
Figures 1 and 2. As can be seen, it was hypothesized that 
counselors' background characteristics would be related to their 
education and training as well as to their work setting and job 
title. It was hypothesized that both education and work affect 
the test use roles of counselors. Education and training were 
believed to affect counselors' knowledge of basic measurement 
concepts and, also, whether or not the counselors have read test 
use guidelines. Education and training, work setting, knowledge 
of measurement concepts, and reading test use guidelines are 
expected to affect counselors' test use practices. 
Results. 

Descriptive Analysi s. Nearly three quarters of the 
respondents work in an educational setting — 29 percent in 
elementary or middle schools, 24 percent in high schools, and 21 
percent in two-year or four-year colleges and universities. Nine 
percent of the respondents are in private counseling practice; in 
addition, twenty percent of those working in other settings have 
a private counseling practice in addition to their primary job. 
Slightly more than half the respondents (53 percent) give 
"counselor" as their job title; 13 percent of the respondents are 
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counselor educators, 6 percent are counseling administrators, 5 
percent are counseling supervisors, 4 percent are research or 
measurement specialists, and the remainder hold a variety of 
other positions. In this paper, unless otherwise indicated, we 
use the term "counselor" to describe all respondents, not simply 
those who have the job title "counselor." 

Nearly all of the respondents hold an advanced degree; 62 
percent have a master's degree, 7 percent a specialist's degree, 
and 31 percent a doctorate. The most common degree fields are 
counseling and guidance, counseling, psychology, and educational 
psychology. Most respondents report taking one or more 
measurement-related courses, such as Tests and Measurement, 
General Statistics, Individual Appraisal, or Educational Research 
Methods, in the program for their highest degree. In addition, 
approximately half (51 percent) of these counselors have taken 
one or more in-service courses or workshops in testing or 
measurement since obtaining their highest degree. 

The average age of the respondents is 47. Most of the 
respondents (95 percent) are White; 58 percent are females and 42 
percent are males. 

Test Use Patterns. The average respondent reports spending 
about six hours a week selecting, administering, and/or 
interpreting tests. Seventy three percent of the respondents say 
tests are very important or important in helping them carry out 
their work. 
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Overall, 62 percent of the respondents indicate they have 
some responsibility for test selection (30 percent select tests 
for a school or a school system and 32 percent select tests for 
use in their own counseling) ; 82 percent of the respondents 
indicate they have some responsibility for test administration 
and 91 percent say they had some responsibility for test 
interpretation . 

Confidence in Ability to Use Tests and to Use Measurement 
Concepts. Sixty nine percent of the respondents say they are 
highly confident of their ability to use test results in 
counseling. The respondents are also highly confident of their 
ability to select tests (67 percent), administer tests (90 
percent) and interpret tests (72 percent) . 

The respondents were asked to rate their confidence in their 
ability to understand and use important concepts in testing and 
measurement. Seventy two percent are highly confident of their 
ability to use test norms and standardised scores; 67 percent are 
highly confident of their ability to use statistics such as mean, 
standard deviation and correlation; 59 percent are highly 
confident of their ability to use test reliability and test 
validity information; and 58 percent are highly confident of 
their ability to use error of measurement information. 

Test Selection. As indicated earlier, the respondents are 
less likely to have test selection responsibilities than to have 
responsibilities for test interpretation or test administration. 
For each test selection, administration, and interpretation 
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practice, the respondents were asked if this is something that 
they always or almost always do, do more than half of the time, 
do less than half of the time, or do seldom or never. Mean test 
selection responses are shown in Table 1. The test selection 
practices covered in the survey are all fairly commonly followed 
by the individuals who have responsibilities in this area. 

Test Administration. Many counselors have responsibilities 
for test administration as well as for test selection. The test 
administration practices of the respondents are shown in Table 2. 
The most common test administration practice identified in this 
survey is explaining the reason for giving the test (89 percent 
of the respondents say they always or almost always do this) . The 
least common test administration practice is altering 
administration procedures so people will feel more at ease taking 
the test (54 percent say they seldom or never do this) . 

The respondents were also asked if they use any computer- 
administered tests; 29 percent say they do so. Those individuals 
who use both computer-administered and paper-and-pencil tests 
were asked to indicate which test administration format they 
prefer. Sixty percent say they prefer paper-and-pencil tests 
while forty percent prefer computer-administered tests. 

Test Interpretation. As noted earlier, counselors are more 
likely to have test interpretation responsibilities than 
responsibilities for test administration or test selection. Mean 
responses are shown in Table 3. The most common test 
interpretation practice among the respondents who have any te&t 
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interpretation responsibilities is reading the test manual to 
find out any limitations of the instrument (65 percent say they 
always or almost always do this) * The least common test 
interpretation practice is using a test in a way not specified in 
the manual (84 percent say they seldom or never do this) . 

The respondents were also asked if they use computer- 
generated test interpretations; those who do so were asked the 
amount of confidence they have in these interpretations and the 
frequency with which they modify these interpretations, sixty 
two percent of the respondents make some use of computer- 
generated test interpretations. Of these, 24 percent have high 
confidence in the interpretations and 69 percent have moderate 
confidence in them. Thirty two percent of the respondents who 
use computer-generated test interpretations say they seldom or 
never modify the computer-generated interpretation while an 
additional 25 percent say they modify them less than half the 
time. 

Familiarity with Test Use Guidelines. Because counselors* 
test use practices may be affected by their knowledge of test use 
guidelines, the respondents were asked if they had read the Code 
of Fair Testing Practices in Education or the AACD/AMECD 
statement on Responsibilities of Users of Standardized Tests. 
Forty two percent of the respondents had read the Code and 52 
percent had read the RUST statement; 59 percent had read one or 
both of these test use guidelines. 
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Relational Analysis. Regression analysis was used to answer 
Question 1 ~ " what variables are most strongly associated with 
having read either or both of the test use guidelines (the Code 
of Fair Testing Practices in Education and/or Responsibilities of 
Users of Standardized Tests)?**. The results are shown in Table 
4. 

The regression variables account for 17 percent of the 
variance in having read these test use guidelines. All else being 
equal, having read the Code and/or RUST guidelines is 
significantly associated with in-service tests and measurement 
training, confidence in knowledge of basic measurement concepts, 
having a job that involves test selection, and sex. 

Finally the variables associated with good test use 
practices were examined to answer Question 2. Good test selection 
practice was defined as including all four test selection items 
(read reviews, examine specimen sets, read test 
manuals/publisher* s information, and review test content). Good 
test administration practice was defined as explaining the reason 
for giving the test, explaining how test results will be used, 
modifying administration procedures to accommodate people with 
disabilities, and explaining how test takers can review their 
scores and ask questions about the results. Good test 
interpretation practice was defined as reading the manual to find 
out the limitations of the test, reading the technical section of 
the manual, taking into account any differences between test 
takers and the group on which the test was normed, and obtaining 
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additional information about test takers to support or refute 
test results. The regressions for good test selection, 
administration, and interpretation practices are shown in Tables 
5, 6, and 7. The regression variables account for 12 percent of 
the variance in test selection practices, 6 percent of the 
variance in test administration practices, and 33 percent of the 
variance in test interpretation practices. 

Good test selection practices are, all else being equal, 
significantly associated with in-service tests and measurement 
training, having read the Code of Fair Testing Practices and/or 
Responsibilities of Users of Standardized Tests, and work setting 
(See Table 5) . 

Good test administration practices are significantly 
associated with work setting, confidence in knowledge of 
measurement concepts, and reading ''.he Code or the RUST statement 
(See Table 6) . 

Good test interpretation practices are associated with 
confidence in knowledge of basic measurement concepts, job title, 
in-service training in tests and measurement, reading the Code of 
Fair Testing Practices and/or the Responsibilities of Users of 
Standardized Tests, work setting, and degree (See Table 7). 

While the regression results should not be interpreted as 
implying causality, they do show that good test use practices are 
consistently and significantly associated with counselors having 
read test use guidelines. 
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Discussion and Conclusions. 

The survey results show that, in general, the counselors 
responding to this survey adhere to many important test use 
guidelines. When selecting tests, seventy two percent always or 
almost always read test manuals or publisher's information and 
the same proportion always or almost always examine specimen sets 
of possible tests, practices recommended for test users in the 
Code of Fair Testing Practices, A. 3 and A. 6. when 
administering tests, 89 percent always or almost always explain 
the reason for giving the test and 82 percent always or almost 
always explain how test results will be used, two very important 
parts of providing proper orientation before test administration 
(AACD/AKECD, Responsibilities of Users of Standardized Tests, 
v.l; AACD, Ethical Standards, C.8). When interpreting tests, 84 
percent say they seldom or never use a test in a way not 
specified in the manual showing compliance with the AACD/AMECD 
guideline that "use of a test for a purpose for which it was not 
designed may constitute misuse. The nature of the validity 
evidence for a test depends upon its use." (Section V. B. ) In 
interpreting test results, 65 percent always or almost always 
read the manual to find out the limitations of the test, since 
the AACD/AMECD statement on Responsibilities of Users of 
Standardized Tests says in Section VIII, Test Interpretation, 
H Proper interpretation requires knowledge about the test which 
can be obtained by studying its manual and other materials ... ; 
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no one should undertake the Interpretation of scores on any test 
without such study. 1 * 

However, in other areas there are some causes for concern. 
For example, AACD Ethical Standards say "The member must proceed 
with caution when attempting to evaluate the performance of 
minority group members or other persons who are not represented 
in the norm group on which the instrument was standardized. " Yet 
45 percent of the individuals responding to this survey say they 
do not routinely consider this and 12 percent say they seldom or 
never consider it. A second cause for concern is the report by 
57 percent of the counselors responding to this survey that they 
do not routinely modify test administration procedures to 
accommodate people with disabilities; indeed, 24 percent say they 
seldom or never make such accommodations. However, Federal 
Regulations (Department of Health, Education and Welfare, 1977) 
require that test users assure themselves "that the tests are 
administered in such a way that the results reflect the abilities 
of handicapped people rather than the presence of the handicap.** 

Counselors* use of computer-generated test interpretations 
also raises some concerns. This is a relatively new aspect of 
test use and many counselors are uncertain about how to proceed. 
The 1988 revision of AACD's Ethical Standards recognized this and 
included helpful material. For example, the Standards now 
recognize **the possibility that computer technology may be 
misused by counselors who rely on computers to compensate for 
their lack of training and experience in test interpretation** 
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(Allen et al., 1988). However, 32 percent of the AACD members 
responding to this survey say they seldom or never change 
computer-generated test interpretations. 

A final area of concern comes from the tendency of some 
counselors to use test scores alone in decision-making rather 
than seeking out other information to provide a more complete 
picture of the individual. As the AACD Ethical Standards point 
out, " Test results constitute only one of a variety of pertinent 
sources of information for personnel, guidance, and counseling 
decisions. " Therefore, responsible test use requires that 
counselors "use all available evidence to infer the validity of 
an individual's score. A single test score should not be the 
sole basis for a placement or selection recommendation** 
(Responsibilities of Users of Standardized Tests, VII, A.l.b.). 
However, 44 percent of the AACD members responding to this survey 
indicated that they do not always or almost always obtain other 
information about individuals to support or refute test results. 
Over-reliance on test information can result in unfairness, as is 
discussed in the recent report of the National Commission on 
Testing and Public Policy (1990). This report points out that 
**when test results alone are used in selection, misclassif ication 
falls disproportionately on minority groups** and recommends that 
test scores not be used alone to make decisions about 
individuals, groups or institutions. 

Moreover, these results may be a somewhat optimistic picture 
of test use. As noted earlier, the respondents are not 
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representative of the group sampled. Practicing counselors are 
under-represented and the respondents are more highly educated 
than the typical members of these two AACD divisions. In 
addition, the questionnaire results are self -report data that may 
have been affected by social desirability. For example, 
counselors may know that they should always explain the reason 
for giving the test and, therefore, answer that they always do so 
even though their actual test use practice is less ideal. 

The regression analyses show how important it is for 
counselors to be familiar with test use guidelines. Good test use 
practices in all three areas are consistently and significantly 
associated with counselors having read the Code of Fair Testing 
Practices and/or the Responsibilities of Users of Standardized 
Tests . 

The regressions also point out the importance of in-service 
training in tests and measurement for counselors. Counselors who 
have had such training are more likely to engage in good test 
selection and good test interpretation practices. Finally, the 
regressions emphasize the importance of counselors' understanding 
basic measurement concepts, such as error of measurement. 
Counselors who are confident of their measurement knowledge are 
more likely to use good test administration and test 
i nterpret at ion pract ices . 
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Table 1 

Test Selection Practices of Respondents 

(n - 405) 

Scale: 1 - Seldom or never, 2 » Less than half the time, 
3 * More than half the time, 4 ■ Always or almost always 

Mean 



Read the test manuals or publisher's information 3.64 

Examine specimen sets of possible tests 3.59 

Review the content of possible tests 3.58 

Read reviews of possible tests 3.36 

Table 2 



Test Administration Practices of Respondents 

(n» 547) 

Scale: 1 * Seldom or never, 2 - Less than half the time, 
3 ■ More than half the time, 4 - Always or almost always 

Mean 



Explain the reason for giving the test 3.86 

Explain how test results will be used 3.78 
Explain to test takers how to review their scores 

and ask questions about results 3.18 
Modify administration procedures to accommodate 

the physically handicapped 2.74 

Explain the best strategies for taking a tes* 2.65 

Provide practice materials before giving a test 2.55 
Alter administration pre *dures so people will 

feel more at ease taking the test 1.86 
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Table 3 

Test Interpretation Practices of Responding Counselors 

(n- 597) 

Scale: 1 * Seldom or never, 2 * Less than half the tine, 
3 - More than half the tine, 4 * Always or alnost always 

Mean 



Read the nanual to find out the limitations of the test 3.49 

Obtain other Information about Individuals to support 

or refute test scores 3.20 

Take Into account differences between the Individuals 

being tested and those on whom the test was normed 3.07 

Read technical section of a test manual 3.01 

Take Into account Individuals' familiarity 

with test content 2.80 

Use different norms for males and females 2.75 

Take Into account Individuals' test taking skills 2.75 

Explain significance of different norm groups 2.71 

Use different norms for different racial/ethnic groups 2.34 

Disregard norms when testing people from foreign 

language backgrounds or other cultures 2.16 

Compare test results by sex and race/ethnicity 1.98 

Collect data to develop local norms 1.92 

Set or use a passing or cut score 1.56 

Use a test in a way not specified in the manual 1.21 
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Table 4 



Variables Associated with Having Read 
the Code of Fair Testing Practices in Education 
and/or Responsibilities of Users of Standardized Tests 



Standardized t-Statistic 
Regression Wt. 



Sex (Male) 


.11 


2.55* 


Degree ( Doctorate ) 


.05 


0.97 


Work Setting (Education) 


-.04 


-1.03 


Job Title (Counselor) 


-.05 


-1.13 


Importance of Tests 


.03 


0.64 


Select Tests 


.11 


2.31* 


Administer Tests 


.03 


0.58 


Interpret Tests 


-.02 


-0.43 


Knowledge of T&M Concepts 


.12 


2.76** 


In-Service T&M Course 


.20 


4.73*** 



R 2 - .17 



* p ■ .05 
** p « .01 

*** p m .001 

Table 5 



Variables Associated with Good Test Selection Practices 



Sex (Male) 

Degree (Doctorate) 

Work Setting (Education) 

Job Title (Counselor) 

Importance of Tests 

Knowledge of T&M Concepts 

In-Service T&M 

Have read Code/RUST 



Standardized t-Statistic 
Regression Wt. 



.00 -0.04 

.04 0.72 

.11 2.20* 

.07 -1.20 

.06 1.22 

.08 1.42 

.17 3.22*** 

.14 2.58** 



R 2 - .12 

* p ■ .05 
** p ■ .01 
*** p * .001 
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Table 6 



Variables Associated with Good Test Administration Practices 



Standardized t-Statistic 
Regression wt. 



Sex (Male) 


.01 


0.19 


Degree (Doctorate) 


-.08 


-1.47 


Work Setting (Education) 


.15 


3.01** 


Job Title (Counselor) 


.02 


0.29 


Importance of Tests 


-.09 


-1.76 


Knowledge of T&M Concepts 


.15 


2.85** 


In-Service T&M 


.01 


0.10 


Have read Code/RUST 


.11 


2.05* 



R 2 - .06 



* p - .05 
** p - .01 
*** p - .001 



Table 7 



Variables Associated with Good Test Interpretation Practices 

Standardized t-Statistic 
Regression Wt. 



Sex (Hale) 

Degree (Doctorate) 

Work Setting (Education) 

Job Title (Counselor) 

Importance of Tests 

Knowledge of T&M Concepts 

In-Service T&M 

Have read Code/RUST 

R 2 - .33 



-.08 -1.91 

.13 2.88** 

-.11 -2.95** 

-.17 -3.94*** 

.08 2.12* 

.23 5.54*** 

.13 3.38*** 

.13 3.23*** 



* p * .05 
** p m .01 

*** p * .001 
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Figure 1 

Conceptual Model - Variables Associated with Having Read Test Use Guidelines (Ql) 
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Figure 2 

Conceptual Model - Variables Associated With Good Test Use Practices (Q2) 
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